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ABSTRACT 

An evaluative study was done of the Teen Outreach 
Program, a nat ional ,^ mul ti-s i te effort to reduce teenage pregnancy, 
school failure, and dropout. The study sought to identify the 
critical **active ingredients" of the program responsible for its 
success. The study was based on analyses of data collected at 114 
different sites nationally, from September 1986 through June 1989, 
involving 1,950 student participants and 2,257 carefully matched 
comparison students. Outcomes for students were assessed in terms of 
the number of problem behaviors experienced by Teen Outreach and 
comparison students at a given site, both prior to entry into the 
program and after participation in it for 1 year. A cumulative 
analyr'-; of the data produced a summary of what is currently known 
about the conditions under which Teen Outreach is most effective and 
the aspects of the program that account for its success. These 
findings indicated that important program components were volunteer 
service by participants and classroom climate. Program structure 
(during versus after school) did not affect outcome. Student 
characteristics of gender, race, ethnic status, and socioeconomic 
status do not affect outcomes. Teen Outreach was found to be most 
effective with middle school students. A fact sheet, one figure, and 
a brief program description are included. (JB) 
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Teen Outreach Process Evaluation - January 1991 

1. 



Overview and Approach 

After several consecutive years of documented success in reducing rates of teenage 
pregnancy, and school failure and dropout (Philliber & Men, in press), the Teen 
Outreach Program began broadening its evaluation in the 1986-87 school year to try to 
identify the critical 'aaive ingredients' of the program that are responsible for its unusual 
record of success. Our- strategy has been to identify student, facilitator and program 
characteristics that are related to the success of Teen Outreach at its most successful 
sites. Because Teen Outreach is a national, multi-site program, we are able to examine 
in detail which sites are most (and least) effective, and use this information to learn 
more about how and why the program works. In short, we have taken a rare step in 
program evaluation research by moving beyond asking "Does it work?" to consider the 
more informative questions: "What works?" "With whom?" and, "Under what conditions?" 



The res\ilts reported below are based upon analyses of data collected at 114 
different sites nationally, from September 1986 through June 1989, involving 1950 
students in Teen Outreach, and 2257 carefully matched comparison students. Outcomes 
for students were assessed in terms of the number of problem behaviors experienced by 
Teen Outreach and comparison students at a given site, both prior to entry into the 
program, and after participating in it for one year. These problem behaviors include 
school failure, school dropout, suspension from school and experiencing or causing a 
pregnancy. In our analyses, we are careful to statistically account for important 
differences between sites (such as the number of problems experienced by students prior 
to entering Teen Outreach), and for changes that appear to be occurring not just in Teen 
Outreach students but in the larger student body at a school.^ This strategy and our 
initial experience in examining these questions is already being documented as the state- 
of-the-art in evaluation research in major scholarly journals (Allen, Philliber & Hoggson, 
1990). This report summarizes what we have learned to date from our efforts to 
evaluate the processes by which Teen Outreach is working. 



^ This is done using a hierarchical regression model in which the total number of 
student problem behaviors at exit from Teen Outreach are predicted first by the number 
of problem behaviors at entry into the program, and second by the change in 
problematic behavior which occurred from entry to exit in the comparison group of 
students at each site. Only after these effects arc accounted for statistically do we 
consider the effect of other program features. Papers describing these analytic strategies, 
and results of analyses for data from individual academic years are available from the 
Association of Junior Leagues International, Inc. 
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Our evaluation efforts have been cumulative in nature, with each year's data being 
used to answer questions raised in prior years. What follows is a summary of what we 
have currently know about the conditions under which Teen Outreach is most effective, 
and the aspects of the program that are most closely related to its unprecedented record 
of success in preventing school dropout and teenage pregnancy. 

Caveats 

Several important caveats need to be kept in mind when interpreting these (and 
any other similar) process evaluation data« Our analytic strategy does not prove that 
program features that are related to program success at a site actually caused that site's 
success; rather our data tell us which factors are consistently associated with successful 
sites across the country. A second important caveat is that our approach can only detea 
relationships between program success and program features when a feature varies 
significantly aaoss sites. Because we rely upon comparing different sites to each other» 
we cannot determine the importance of a feature if it is uniformly (or nearly uniformly) 
used across all sites. For example, if the curriculum were very consistently used at all 
sites, it would be virtually impossible to determine whether it was related to the success 
of the program. Only if we had sites that both did and did not use the curriculum very 
mr -"h could we determine its importance. 

M^ior Findings 

The findings presented below are divided into findings about four major aspects of the 
implementation of Teen Outreach: the efficacy of individual components of the program; 
the importance of how the program is structured; the role of student demographic 
characteristics; and the role of facilitator demographic characteristics. 

Program Components 

Volunteer service. One of the most consistent findings of three years of process 
evaluation efforts has been that the volunteer component cfthe Teen Outreach Program 
appears very closely linked to its success. Our first look at the volunteer component 
suggested that the number of hours spent volunteering was slightly related to program 
success. Since then, we have recognized that the nature of the volunteer work actually 
performed, and how it is received by students, appears far more important than the 
number of houn of work performed. Specifically, volunteer work that engages students, 
and that they feel they had a choice in selecting, was very significantly associated with 
program success. These findings are based upon student's descriptions of the volunteer 
work; facilitator's perceptions of the volunteer work were not good predirtors of program 
success. It thus appears to be quite important to get the impressions of young people 
performing the volunteer work in order to determine whether the work is likely to be a 
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3. 

positive experience for them. Analyses of our most recent process data also suggest that 
volunteer work performed in human service settings was more likely to be associated 
with the program's success in reducing pregnancy and dropout, than was volunteer work 
performed in school. 

Curriculum use. The extent to which individual sites used the Teen Outreach 
curriculum has been marginally related to the success of the program at those sites. It 
was in Teen Outreach programs geared toward middle school students that use of the 
curriculum was most likely to be related to positive program outcomes. It is possible 
that the curriculum does not appear to be more important in part because virtually all 
sites use a significant portion of it. Thus we are left trying to distinguish sites that use a 
lot of the curriculum with sites that use a moderate amount of it. 



Classroom characteristics. Teen Outreach classrooms in which students feel the 
dimate is emotioruUfy supportive and in which students (not facilitators) do a ffeat deal of ' 
the talking were detfffy associated with the success of the proffom. However, these findings 
were strongest in middle school classrooms, and were more marginal for high school 
classrooms. It did not appear to matter whether students formed a strong individual 
relationship or bond with their facilitator...as long as the facilitator was able to create a 
classroom environment in which students were not worried about being ""put down'' and 
could discuss fears and uncertainties. 

Program Structure 

We have repeatedly asked whether it makes a difference whether Teen Outreach is 
offered during vs. after school^ and whether it is offered for credit, or as a not-for-credit 
extracurricular activity. No effects of these structural features have ever been detected. 
A marginal relationship was found in one year for the program to fare better in small to 
medium-sized towns than in large urban areas. However, given that the program has 
remained clearly successful even in years in which it was predominantly offered in large 
urban centers, it is clear that this is not an important factor related to the program's 
success. Thus far, therefore, Teen Outreach seems robust, in that where it is offered has 
not been ag^uficantfy related to its success. 

Characteristics of Sftidents 

We have examined numerous characteristics of the students who participate in Teen 
Outreach. Across different sites, a wide range of students participate in the program. 
Student characteristics which are aol related to the success of the program include students' 
gertder^ racial or ethnic status, and sodo-economic status. There has been some evidence 
to suggest that the program may be more effective with students who are from single- 
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parent families, although this effect appears slight. 

The one student characteristic that did appear related to program success in our 
evaluations was age. Teen Outreach has appeared to be most effective with older children. 
Evidence thus far supports the conclusion that the Teen Outreach program is successful 
with younger (e.g. middle school) students when the classroom portion of the program 
functions well. However, it should be noted that these findings are based upon 
evaluations of the program in the years in which it was first being offered to large 
numbers of middle school students, and thus may not have yet been as refined to the 
needs of that age group as it was to those of high schooi*age students. 

Facilitator Characteristics 

In trying to identify characteristics of the most effective Teen Outreach facilitators, we 
have had relatively little success thus far. Although there has been some slight signs that 
facilitators who were more educated, and who had more children of .heir own may have' 
lead slightly more successful programs, the jmUngs indicate that fadlitators ofaB mces, 
with a wide range of levels of education, and from very <Ufferertt backffounds have 
successfuBy lead Teen Outreach proffrmts. 

Conclasions 

The three years of process evaluation findings summarized above complement the 
nationally recognized efforts evaluating the outcomes of the program (Philliber & Allen, 
in press) by providing a tremendous amount of information about how and why this 
innovative prevention program works. Using data from later years of the program 
replication, we are now beginning to examine more intensively the aspects of the 
program which have appeared most promising thus far. Although this report summarizes 
only our initial efforts at evaluating the processes by which Teen Outreach works, the 
amount of information provided is virtually without parallel among innovative prevention 
programs, or even among human service programs in general. As a result of these 
efforts, we now have strong evidence of the value of volunteer community service for 
young people, as well as important information about the structure of the program in 
which this service should optimally be imbedded. These findings not only guide the 
further development of the Teen Outreach Program, but also yield valuable insights 
about the critical ingredients of successful preventive interventions with youth in our 
society. 
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THE EVALUATION OF TEEN OUTREACH 
1984 to 1991 



The evaluation of Teen Outreach began in the 1984-85 school year and has now covered 7 
years, through 1990-91. 

During that time, data have been gathered from! 

*^ 3,674 Teen Outreach students, and 

4,202 comparison students, nationwide and in Canada. 

During the last tliree of these years, a portion of these students were in sites where true 
random assignment of Teen Outreach and control students was possible. The random 
assignment data set includes: 

** 472 Teen Outreach students, and 
** 496 comparison students. 

In the entire 7 year sample, the following results have been obtained: 

a 5% lower nxi of course failure in school; 
an 18% lower rate of school suspension; 
a 33% lower rate of pregnancy; and 

a 50% lower rate of school dropout among Teen Outreach students than among 



The differences in these rates between Teen Outreach and comparison students are 
statistically significant and participation in Teen Outreach is significantly related to each 
outcome when controls are introduced for race, gender, grade, mother's education, living 
arrangement, and pre-program levels of each of these indicators. In other words, this 
impact of Teen Outreach is net of the impacts of these other factors. 

These same results in the random assignment subsample show: 

a 32% lower rate of course failure in school; 
a 37% lower rate of school suspension; 
a 43% lower rate of pregnancy; and 

a 75% lower rate of school dropout among Teen Outreach students than among 
THE ASSOCIATION OF ^ Hindomly assigned group of control students. 
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INTERNATIONAL INC. 

660 FIRST AVENUE 

NEW YORK. NY. 10016.3241 
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comparison students. 
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\, y TEEN OUTREACH PROGRAM 

The Teen Outreach Ptogram (TOP) was established in 1978 by Brenda HosteUer, director of pregnancy prevention 
orograms in the St. Louis Public Schools. In 1981 , the Junior League of St. Louis and the Danforth Foundation 
began spomcring the program, whose goai was to decrease the rate of adolescent pregnancy and increase the rate 
of high-school graduation for at-risk teens. In 1984, the Junior League of St. Louis initiated a national 
demonstration effort funded by the Charles Ste'vart Mott Foundation. Since 1987 the Association of Junior 
Leagues International Inc. (AJLI) has been responsible for the national program, widi ftinding from the Charles 
Stewast Mott Foundation, the Lila Wallace-Reader's Digest Fund, the Smith Richardson Foundation, the Carnegie 
Corporation of New York, and the Stuart Foundations. 

The Teen Outreach Program expanded from eight Junior League sponsors with nine classroom sites in 1984 to 
a total of 33 Junior Leagues and other conununity-based sponsors and 95 classroom sites in the 1992-93 school 
year. AJLI is currently developing state and community r: ■ >dels to iastitutionalize TOP and replicate it on a wider 
basis. A sute model for TOP is being developed in Calitomia, and preliminary discussions about sute nwdels 
are also underway in two other states. Additionally, AJLI is working widi community sponsors in seven 
conununities (Atlanta; Bronxville, NY; New Orleans; Reno; Roanoke, VA; Charleston, WV; and Seattle) to 
develop a community/school pannership which will focus on implementing the program on a district-wide basis. 

Teen Outreach began as a comprehensive program to help adolescents develop a"l[>ositive self-image, concrete life 
management skills, and future goals, though the marketing strategy identified it primarily as a program aimed 
at preventing adolescent pregnancy. Over die past six years, its focus has been on fostering positive development 
in at-risk youth. 

In every Teen Outreach Program, a group experience, a facilitator/student relationship, and a volunteer experience 
build self-esteem and individual skills. A unique relationship develops between the program facilitators and the 
femaleand male adolescents who participate in the TOP curriculum in small peer-group settings. The community 
service component enhances the students' sense of self-worth and enables diem to see themselves as valuable, 
cnntributing members of their communities. Both the volunteer component and the classroom discussions have 
been identified as key elements in TOP's success. 

The 'helper-therapy' principle introduced by Rlessman 0965) suggests that helping other people can be dierapeutic 
and can lead to personal growth, particularly for persons in dlsempowered groups. A sease of empowerment is 
engendered by placing students in help-giving raUier than help-seeking roies, such as in hospitals, nursing homes, 
day care centers and other community sites (Bronfenbrenner, 1979; Rappaport, 1987). The volunteer component 
is also based on the premise diat when young people become effective volunteers, a belief that they can succeed 
in attempts to behave competenUy Increases. This belief in turn leads to more persistent effort to perform 
competenUy (Bandura, 1977). 

TOP'S volunteer component is consistent with the notion that education may most effectively proceed in real-life 
settings outside of classrooms (Sarason, 1984). Gottfradson (1985) has noted that outside work may provide 
youUis who do not fit into the mold of traditional schooling an alternative route for success; work may provide 
a legitimate means to meet needs and strengthen adolescents' bond to the social order. Because volunteer 
experiences teach pre-employment skills, yet do not Increase financial independence from parents, they do not 
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weaken parental control, a potential problem with teenage jobs. A successful TOP volunteer component ideally 
offers assignments that actively engage students and choices about their volunteer work. 

TOP links the volumeer program to the school experience; a U.S. Department of Labor study found that work 
experience, when closely coordinated with school, increases both school attendance and grades (Barton & Frazer, 
1980). An emotionally supportive classroom environment in which the students do most of the talking and 
facilitators relate to students in a compassionate and non-judgmental manner has been found to be most successful . 
The Teen Outreach classroom activities increase students' commitment to academic endeavors, to tlie program 
facilitator, to each other, and to TOP itself. Teen Outreach may increase adolescents' attachment and 
commitment to basic societal norms, such as completing school and avoiding delinquent behavior. 

Finally, the Life Options curriculum, because it emphasizes positive decision-making skills, may enhance 
adolescents' skills in dealing with social situations they face. The curriculum presents topics and exercises that 
the faciliutor can use to generate further group discussion. The content and the interactive style of sessions about 
relationships, life planning, peer pressure and family issues help teens learn to think critically about these issues 
and to evaluate their own behavior. TOP participants are helped to develop better communication skills and learn 
to resolve differences of opinion and other conflicts. 

The average Teen Outreach group consists of 15 to 20 females and males. Sessions take place during the daily 
school cuniculum or after school; at some of the schools it may be taken for credit. All Teen Outreach programs 
have a community sponsor, typically a Junior League but sometimes another community agency. The sponsor 
is responsible for 1) securing funding for the program, 2) arranging for and monitoring the volunteer experiences, 
and 3) collaborating with the school system. 

TOP includes a significant evaluation component and maintains a databasic' on participants and comparison 
studints. National data indicate that the program has resulted in statistically significant reductions in teenage 
pregnancies and school failure. In the seven years ending in 1991, Teen Outreach participants averaged a 5 
percent lower rate of course failure, a 18 percent lower rate of school suspension, a 50 percent lower rate of 
school drop out, and a 33 percent lower rate of pregnancy than students in the control group. 

The Teen Outreach Evaluation represents 3,674 students at 60 sites who range in age from II to 21, axvi whose 
averige age is 14.9 yean old. About 40% of the students arc black, 40% are white, and 13% are Latina/o. 
Those in otiicr racial/ethnic groups include mostly native Americans and Asians. 

Two-fifths of the participants come from single parent families; the parents of about one-fifth had less than a high 
school education. Approxunately 1500 students participated m the program in the 1991-92 school year. Some 
students enter TOP on a volunteer basis when they hear of the program; others are targeted by program 
facUitatots or counsclore as at "high risk" for leaving school or becoming pregnant. High risk factors mclude 
having a parent or older sibling who did not graduate from high school or who became pregnant as a teenager. 
At still other schools, facilitators seek out students who are not yet exhibiting negative behaviors but who could 
be at risk. 

TOP'S congruence witii related research undoubtedly contributes to its unusually positive outcomes. In 1987, the 
report of the National Re*earch Council's Panel on Adolescent Pregnancy and Childbearing, Risking the Funire; 
Adolescent Sexuality. Pregnancy and Childbearine . named TOP as one of only three teen pregnancy prevention 
strategies with any documented evidence of reducing pregnancy. The first published academic article on the Teen 
Outreach Program was publUhed in the Am erican Journal of Community Psvch olQg Y >« December 1990, emitied 
"School-Based Prevention of Teenage Prejjnancy and School Dropout: Process Evaluation of the National 
Replication of the Teen Outreach Program.' More recently, TOP's success has been documented in Preventin g 
Adolescent Pregnancy: Mode\ Programs and Evaluations. 
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